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DETAILED ACTION 

Response to Amendments and Arguments 

1 . Applicant's response to the last Office Action, filed June 6, 2004, has been 
entered and made of record. 

2. . Applicant has amended Claims 1-3, 10-12, 18-20 and 29. No Claims have 
been canceled. Claims 1-29 remain pending. 

Drawing Objections 

3. In view of replacement drawings and discussion provided by Applicant the 
previously applied drawing objections are withdrawn. 

Claims Objections 

4. In view of amendments and discussion provided by Applicant, the 
previously applied Claim objections are withdrawn. 

Claim Rejections under 35 USC 112 

5. In view of amendments and discussion presented by Applicant, the 
previously applied Claim rejections under 35 USC 1 12 are withdrawn. 
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Claim Rejections under 35 USC 102 

6. With regard to Applicant's remarks concerning the newly amended 
independent Claims 1,12 and 18, the amendments are not deemed to overcome the 
cited reference of Chen. Applicant has amended the independent claims to include the 
new features of "detecting a plurality of pixels in a base image that represents a first 
view of a scene, wherein the first view is one of at least three views of a scene : 
determining 3-D depth of the plurality of pixels in the base image by matching 
correspondence to a plurality of pixels in a plurality of images, each image representing 
one of the at least three views of the scene wherein each of the at least three views of 
the scene are situated in a non-linear arrangement and are further oriented in a plurality 
of non-parallel planes relative to each other 

Applicant argues that the reference of Chen discloses only 2 views of a scene 
and does not teach that each image represents one of at least three views of the scene . 
Examiner points to the teapot example (Example 4) at the top left of page 34 in Chen. 
Chen discloses the 3D calculation using six stereo images. Therefore the reference of 
Chen still reads on the limitation of at least three views in claims 1,12 and 18. 

Applicant further argues that the images used in the reference of Chen are 
coplanar and therefore do not meet the limitation of wherein each of the at least three 
views of the scene are situated in a non-linear arrangement and are further oriented in a 
plurality of non-parallel planes relative to each other . Examiner points to the example of 
the teapot again (Example 4) at the top right of page 34. The teapot is placed on a 
turntable and rotated to capture images at multiple points of rotation. Therefore the 
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captured images would likewise not be coplanar. Therefore the reference of Chen still 
reads on the amended independent Claims 1, 12 and 18. 

Applicant also bases arguments concerning dependent claims on the subject 
matter discussed in regard to independent claims 1, 12and 18 which have been 
addressed above. 

The rejection of the Claims is therefore maintained and is accordingly made final. 

Rejections Under 35 U.S.C. § 102(b) 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 

form the basis for the rejections under this section made in this Office action: 

(b) the invention was patented or described in a printed publication in this or a 
foreign country or in public use or on sale in this country, more than one year 
prior to the date of application for patent in the United States. 

2. Claims 1,4-9, 12-18, and 21-26 are rejected under 35 U.S.C. 102(b) as being 
anticipated by [Chen, VSMM99] (Chen and Medioni , "A Volumetric Stereo Matching 
Method: Application to Image-Based Modeling", IEEE 1999). 

3. The following is in regard to Claim 1. [Chen, VSMM99] describes a volumetric 
stereo matching method. The method involves the recovery of depth information - 
namely, the disparity surface d(u,v) (see [Chen, VSMM99]: Abstract, sentence 1; page 
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29, right column, last paragraph 1 , sentences 1-3; and page 30, left column, paragraph 
1 , sentences 2-3) - for pixels of a base image representing a view of a scene. The 
method comprises at least the following steps: 

(1 .a.) Detecting a plurality of pixels in a base image that represents a first 
view of a scene wherein the first view is one of at least three views 
of a scene (e.g. capturing images from various viewpoints of the 
scene). It should be apparent from the various stereo images 
shown in [Chen, VSMM99] that the volumetric stereo matching 
method involves this step. Any one of these images can be 
arbitrarily denoted as a "base image". See Also Example 4 on 
page 34, where 6 views of a teapot are used. 
(1 .b.) Determining 3-D depth of the plurality of pixels in the base image 
by matching correspondence to a plurality of pixels in a plurality of 
images each image representing one of the at least three views of 
the scene wherein each of the at least three views of the scene are 
situated in a non-linear arrangement and are further oriented in a 
plurality of non-parallel planes relative to each other . See Also 
Example 4 on page 34, where 6 views of a teapot are used. Chen 
discloses using views of a teapot as it is rotated, the views of which 
would therefore not lie on parallel planes. 



When referring to paragraphs in the cited references, the convention followed here is that the paragraph number is assigned to 
paragraphs of a given column (if applicable) or section, sequentially, beginning with the first full paragraph. Paragraphs that 
carry over to other columns will be referred to as the last paragraph of the column in which they began. 
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Steps (1.a.)-(1.b) are typical of any stereo matching applications. The "matching 
correspondence" is discussed in Section 3.1 of [Chen, VSMM99]. The cross-correlation 
Qu,v,d\s indicative of the matching correspondence between pixels of different images, 
each of which are representative of disparate views of the scene. 

(1 .c.) Tracing pixels (i.e. surface tracing) in a virtual piecewise continuous 
depth surface (i.e. disparity surface d{u.v)) by spatial propagation 
([Chen, VSMM99] section 3.2, paragraph 1 , sentence 4) starting 
from the detected pixels in the base image (i.e. all pixels (u,v) of the 
base image) by using the matching and corresponding plurality of 
pixels in the plurality of images (e.g. seed pixels that satisfy a given 
cross-correlation criterion - [Chen, VSMM99] page 31, right 
column, paragraph 1, sentences 1-2) to create the virtual piecewise 
continuous depth surface viewed from the base image. Please refer 
to Section 3 of [Chen, VSMM99]. 
(1 .d.) Each successfully traced pixel is associated with a depth in the 

scene viewed from the base image. This follows from the fact that 
traced pixels lie on the disparity surface. Again, the disparity 
surface is indicative of the depth of all visible surfaces of a given 
scene (page 29, right column, last paragraph, sentences 1-3). 
In this manner, the a volumetric stereo matching method of [Chen, VSMM99] conforms 
sufficiently to the method proposed by the Applicant in claim 1 . 
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4. The following is in regard to Claim 18. Claim 1 8 recites essentially the same 
limitations as claim 1. (The 3D reconstruction instructions merely implement the depth 
recovery method of Claim 1). Therefore, with regard to claim 18, remarks analogous to 
those presented above relative to claim 1 are applicable. 

5. The following is in regard to Claim 4. As shown above, [Chen, VSMM99] 
discloses a volumetric stereo matching method that adequately conforms to the depth 
recovery method of Claim 1 . The volumetric stereo matching method of [Chen, 
VSMM99] further involves: 

Propagating a front of a virtual piece of a continuous depth surface 
(e.g. a surface front -this terminology is used in sentence 5 of the 
last paragraph of the right column on page 31) to at least one 
neighboring pixel (e.g. each 4-neighborot (u,v) - Fig. 4 of [Chen, 
VSMM99]) starting from the detected pixels (e.g. pixel (u,v)) in the 
base image. 

It should be apparent from the algorithm listed in Fig. 4 of [Chen, VSMM99] that the 
boundary (front) of the disparity surface is propagated. In this manner, the volumetric 
stereo matching method of [Chen, VSMM99] conforms sufficiently to the method 
proposed by the Applicant in Claim 4. 

6. The following is in regard to Claim 21. Claim 21 recites essentially the same 
limitations as claim 4. (The 3D reconstruction instructions merely implement the depth 



Application/Control Number: 09/927,558 Page 8 

Art Unit: 2623 

recovery method of Claim 4). Therefore, with regard to claim 21 , remarks analogous to 
those presented above relative to claim 4 are applicable. 

7. The following is in regard to Claims 5-6. As shown above, [Chen, VSMM99] 
discloses a volumetric stereo matching method that adequately conforms to the depth 
recovery method of Claim 4. Tracing the disparity surface according to the volumetric 
stereo matching method of [Chen, VSMM99] further involves the following: 

(5. a.) Determining when a boundary is reached between two propagating 
fronts of virtual pieces of a continuous depth surface ([Chen, 
VSMM99] page 31 , right column, last paragraph, sentence 5). 
If the boundaries of two propagating fronts coincide then: 

(6.a.) Comparing the cross-correlations of the two propagating fronts 
about the reached boundary. This comparison is performed, for 
example, in the following step shown in [Chen, VSMM99] Fig. 4: 

disparity^',*/') = □</>',</.'□□ tf>',cf:;pd' pcT 
Clearly, a comparison of the cross-correlations, □ u v d ; 'and □ 
u',v',d'[ is tantamount to a comparison of the matching costs of the 
two surface fronts 
(6.b.) The surface front with the greatest cross-correlation prevails 

([Chen, VSMM99] page 31 , right column, last paragraph, sentence 
5). In other words, the propagation of the front with the lower cross- 
correlation is stopped. 
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Step (6.b.) is equivalent to stopping the propagation of the front with the higher 
compared matching cost 2 . In this manner, the a volumetric stereo matching method of 
[Chen, VSMM99] conforms sufficiently to both the methods of Claim 5 and the method 
of Claim 6. 

8. The following is in regard to Claim 22-23. Claim 22-23 recite essentially the same 
limitations as claim 5-6, respectively. (The 3D reconstruction instructions of these claims 
merely implement the depth recovery methods of Claim 5-6). Therefore, with regard to 
claim 22-23, remarks analogous to those presented above relative to claim 5-6 are 
respectively applicable. 

9. The following is in regard to Claim 7. As shown above, [Chen, VSMM99] 
discloses a volumetric stereo matching method that adequately conforms to the depth 
recovery method of Claim 1 . Tracing the disparity surface according to the volumetric 
stereo matching method of [Chen, VSMM99] further involves the following: 

Propagating a front of a virtual piece of a continuous depth surface 
to at least one neighboring pixel surrounded by a predefined size 
window in the continuous depth surface. According to the 
volumetric stereo matching method of [Chen, VSMM99], the 
normalized cross-correlation (NCC) over a window (e.g. one with 

2 To see this recall how the Applicant defines matching cost (page 9, lines 1 0- 1 5 of the Applicant's disclosure and, in 
particular, equation (8) on page 17). The matching cost is low for pixels that are likely matches and is high for unlikely 
matches. The relationship between the matching cost and cross-correlation (NCC) is shown in the Applicant's equation (8). 
Clearly, a high NCC results in a low matching cost E(g) and indicates a likely match, whereas a low NCC results in a high 
matching cost and indicates an unlikely match. Therefore, by stopping the propagation of the front with the lower NCC, one 
has effectively stopped the propagation of the front with the higher matching cost 
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dimensions □□□and centered at pixels (u,v) and (u+d,v)) is used 
as the similarity measurement. See Section 3.1 of [Chen, 
VSMM99]. 

In this manner, the a volumetric stereo matching method of [Chen, VSMM99] conforms 
sufficiently to the method of Claim 7. 

1 0. The following is in regard to Claim 24. Claim 24 recites essentially the same 
limitations as claim 7. (The 3D reconstruction instructions merely implement the depth 
recovery method of Claim 7). Therefore, with regard to claim 24, remarks analogous to 
those presented above relative to claim 7 are applicable. 

1 1 . The following is in regard to Claim 8-9 and 25-26. As shown above, [Chen, 
VSMM99] discloses a volumetric stereo matching method that adequately satisfies the 
limitations of Claims 7 and 24. The limitations set forth in Claims 8-9 and 25-26 were 
addressed previously in the discussion above relating to Claims 5-6 and 22-23. The 
details will not be repeated here. 

12. The following is in regard to Claim 12. [Chen, VSMM99] discloses a volumetric 
stereo matching method that includes the following: 

(12. a.) Providing a plurality of seed pixels that represent 3-D depth of the 
plurality of pixels in the base image view of a scene (i.e. seed 
voxels - [Chen, VSMM99] page 31 , right column, paragraph 1). 
Seed voxels are determined by matching correspondence to a 
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plurality of pixels in a plurality of images representing a plurality of 
views of the scene, wherein the base image is one of at least three 
views of the scene, each of the at least three views of the scene 
being situated in a non-linear arrangement and are further oriented 
in a plurality of non-parallel planes relative to each other (i.e. 
ensuring that the cross-correlation Uu,v,d between the views 
satisfies Uu,v,dnt1 , t1 being a predetermined threshold - [Chen, 
VSMM99] page 31, right column, paragraph 1, lines 3-4). See also 
example 4 on page 34 with the 6 rotating views of the teapot. 
(12.b.) Tracing pixels in a virtual piecewise continuous depth surface by 
spatial propagation starting from the provided plurality of seed 
pixels in the base image by using the matching and corresponding 
plurality of pixels in the plurality of images to create the virtual 
piecewise continuous depth surface viewed from the base image, 
each successfully traced pixel being associated with a depth in the 
scene viewed from the base image. The surface tracing was 
treated above with respect to step (1 .a). See also [Chen, VSMM99] 
right column, paragraph 2 and Fig. 4. 
13. [Chen, VSMM99] presents the volumetric stereo matching method entirely as an 
algorithm and forgoes any mention of possible hardware implementations. However, the 
volumetric stereo matching method is shown to have been executed on a computer 
(e.g. SGI workstation - [Chen, VSMM99] page 34, left column, paragraph 2). In this 
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case, item (12. a.) would be implemented in computer-readable instructions, say as a 
"image matching module", and item (12.b.) would be implemented in computer-readable 
instructions, say as a "propagation module". The "image matching module" and 
"propagation module" would presumably be resident (electrically coupled) in some 
memory which is, in turn, coupled to a central processing unit (CPU). In this manner, a 
computer implementation of the a volumetric stereo matching method of [Chen, 
VSMM99] would sufficiently conform to the image processing system of Claim 12. 

14. The following is in regard to Claim 13. As shown above, [Chen, VSMM99] 
suggests a computer implementation of a volumetric stereo matching method that 
adequately satisfies the limitations of Claims 12. The disclosed volumetric stereo 
matching method operates on photographic images (e.g. [Chen, VSMM99] Figs. 2 or 8). 
This necessitates some means of capturing images, such as a camera(s) (see [Chen, 
VSMM99] page 34, left column, paragraph 1). In order for the CPU to process the 
captured images, the camera would have to be somehow electrically coupled (e.g. via 
some camera interface) to the CPU. Therefore, the features of the image processing 
system proposed in Claim 13 are inherent to a computer-based implementation of the 
volumetric stereo matching method of [Chen, VSMM99]. 

15. The following is in regard to Claim 14. As shown above, [Chen, VSMM99] 
suggests a computer implementation of a volumetric stereo matching method that 
adequately satisfies the limitations of Claims 12. As mentioned above the "image 
matching module" and "propagation module" are resident on a memory. Modern 
memory is implemented as an integrated circuit. All modern CPUs are implemented as 
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integrated circuits. In this manner, a computer-based implementation of the a volumetric 
stereo matching method of [Chen, VSMM99] would adequately satisfy the limitations of 
Claim 14. 

16. The following is in regard to Claims 15-17. Claim 15-17 recite essentially the 
same limitations as claim 5-7, respectively. Therefore, with regard to claim 15-17, 
remarks analogous to those presented above relative to claim 5-7 are respectively 
applicable. 

Rejections Under 35 U.S.C. § 103(a) 

17. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 102 of this title, if the differences between the 
subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was 
made. 

18. Claims 27-28 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
[Chen, VSMM99]. 

19. The following is in regard to Claim 27. As shown above, [Chen, VSMM99] 
discloses a volumetric stereo matching method that adequately satisfies the limitations 
of Claims 24. Although [Chen, VSMM99] does not explicitly mention associating the 
normalized cross-correlation (NCC) with a matching cost, the usage of the NCC in 
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[Chen, VSMM99] suggests that the two are closely related and could indeed be used 
interchangeably. As mentioned above, [Chen, VSMM99] uses the NCC, Ou,v,d within 
windows associated with each of the propagating fronts (e.g. windows centered about 
the voxel {u,v,dl on the surface fronts - [Chen, VSMM99] page 31 , left column, lines 1-3 
and right column, paragraph 2, sentence 1 ) to determine which of two coincident 
surface fronts to continue to propagate ([Chen, VSMM99] page 31, right column, last 
paragraph, sentence 5). Furthermore, the NCC is indicative of the "correctness" of a 
match ([Chen, VSMM99], Section 3.2, paragraph 1, sentence 1). All of this is suggests 
the that the NCC provides an adequate measure of the matching cost of the 
propagating surface fronts. Therefore, it would have been obvious to one of ordinary 
skill in the art, at the time of the applicant's claimed invention, to relate the matching 
cost of the propagating surface fronts to the computation of the NCC or designate the 
NCC as the matching cost. In either case, the motivation for doing so would have been 
to utilize the cross-correlative properties of the NCC to evaluate the degree to which 
voxels, comprising the surface fronts, represent correct matches. 
20. The following is in regard to Claim 28. As shown above, [Chen, VSMM99] 
discloses a volumetric stereo matching method that, when modified in the manner 
suggested above, adequately satisfies the limitations of Claims 26. [Chen, VSMM99] 
suggests rectification of at least one pair of images corresponding to the base view of 
the scene and at least one of the other views (e.g. "reference views"). See [Chen, 
VSMM99] Section 2. Therefore, it would have been obvious to one of ordinary skill in 
the art, at the time of the applicant's claimed invention, to further perform rectification of 
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at least one pair of images corresponding to the base view of the scene and at least one 
of the other views (e.g. "reference views"). The result of rectification is that "scanlines 
[epipolar lines] are parallel in each image, and the corresponding ones are collinear 
across the images" ([Chen, VSMM99] Section 2, paragraph 1 , sentence 2 and last 
paragraph). This is known to have the effect of greatly simplifying the problem of finding 
correspondences across stereo images. 

21 . Claims 2-3 and 19-20 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over [Chen, VSMM99], in view of [Zhang94] (Zhang, Deriche, Faugeras, and Luong, "A 
Robust Technique for Matching Two Uncalibrated Images Through the Recovery of 
Unknown Epipolar Geometry", INRIA 1994). 

22. The following is in regard to Claim 2. As shown above, [Chen, VSMM99] 
discloses a volumetric stereo matching method that sufficiently conforms to the depth 
recovery method proposed in Claim 1 . The volumetric stereo matching method of 
[Chen, VSMM99] operates on multiple images (see, for instance [Chen, VSMM99] Fig. 
2 and Fig. 11). That is, the depth recovery according to this method includes the 
following: 

(2. a.) Detecting a plurality of image pixels in a first image corresponding 

to a first view of a scene. 
(2.b.) Detecting a plurality of image pixels in at least a second image 

corresponding to a respective at least a second view of the scene. 
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The at least a second image deviates from the first image as a 
result of camera relative motion. This is typical of stereo matching 
algorithms (see also [Chen, VSMM99] page 34, Example 4). 
As noted above, the volumetric stereo matching method of [Chen, VSMM99], like most 
stereo matching methods, involves determining correspondences between a plurality of 
pixels in a first image and a plurality of pixels of at least a second image. In other words, 
the method of [Chen, VSMM99] further involves: 

(2.c.) Determining a first two-view correspondence between the plurality 
of detected image pixels in the first image and a plurality of 
detected image pixels in one of the at least a second image 
resulting in a first potential match set of candidate image pixels 
between the first image and the one of the at least a second image. 
This process involves the evaluation of the cross-correlation 
Ou,v,das discussed above with regard to step (1.b.). 

23. [Chen, VSMM99] further suggests subsequently computing the projective 
reconstruction from the derived correspondence information in order to reconstruct the 
projective 3D structure of the observed scene ([Chen, VSMM99] Section 4, paragraph 
1). 

24. [Chen, VSMM99] does not, however, expressly show or suggest: 
(2.d.) Concatenating a plurality of two-view match sets to form a first 

multiple-view potential match set: and 
(2.e.) Determining a second multiple-view potential match set that is a 
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refinement of the first multiple-view potential match set . 
25. [Zhang94], on the other hand, discloses a stereo matching method. Stereo 
matching across disparate views of a scene is generally a fundamental step in depth 
recovery. The stereo matching method involves at least the following steps (refer to 
[Zhang94] Section 6.3 on pages 16-19): 

(2.c.) Determining an initial set of correspondences or matches (e.g. a 
first two-view correspondence) between the plurality of detected 
image pixels in the first image and a plurality of detected image 
pixels in one of the at least a second image resulting in a first 
potential match set of candidate image pixels (e.g. candidate 
matches - see, for example, [Zhang94] page 10, Section 5, 
paragraph 1 , lines 1-2) between the first image and the one of the 
at least a second image. 
(2.d.) Removing outliers from the initial set of matches (i.e. determining a 
"multiple-view correspondence" between the plurality of detected 
image pixels in the first image and the plurality of detected image 
pixels in the at least a second image). See, for example, [Zhang94] 
page 18, line 24 3 , in conjunction with the equations for the weights 
w/on pages 17-18. In this manner, the set of matches or 
correspondences thus obtained (e.g. the "multiple-view 
correspondences") represents a refinement of the "first two-view 

3 Lines are counted sequentially from the top beginning with 1 . An equation is treated as a single line, regardless of its length. 
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correspondence", resulting in a "second potential match set" of 
candidate image pixels between the first image and the at least a 
second image. 

(2.e.) The "second potential match set" is based at least in part on a 

computation of reprojection error (e.g. residual n- [Zhang94] page 
17, paragraphs 1-2) for matched pixels that resulted from "the 
difference between the /-th observation and its fitted value [4] " 
([Zhang94] page 17, paragraph 1, lines 1-2). 
26. The teachings of [Chen, VSMM99] and [Zhang94] are combinable because they 
are analogous art. Specifically, both [Chen, VSMM99] and [Zhang94] disclose methods 
for stereo matching. Therefore, it would have been obvious to one of ordinary skill in the 
art, at the time of the applicant's claimed invention, to use the methodology of [Zhang94] 
- in particular, steps (2.d.)-(2.e.) above - to refine the initial set of correspondences 
derived according to the method of [Chen, VSMM99]. It would have been obvious to 
one of ordinary skill in the art, at the time of the applicant's claimed invention, that, in 
applying this refinement process to method of [Chen, VSMM99], the fitted values 
become the reprojected projective points that delimit the projective structure of the 
observed scene (i.e. "matched pixels that resulted from a projective reconstruction of 
the first potential match set"). The motivation for using the method of [Zhang94], in this 
manner, would have been to detect and remove outliers (e.g. bad locations and/or false 



A 



It should be understood that the fitted value in this case is a projection of the matched pixels into some view being evaluated and the 
observation is the respective candidate point correspondences {(mij, m*)}. See [Zhang94] page 17, last paragraph. 
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matches - [Zhang94] page 16, last paragraph) from the initial set of candidate matches 
(e.g. those obtained via step (2.c.) above). See [Zhang94] Section 6.3. 

27. The following is in regard to Claims 19. Claim 1 9 recites essentially the same 
limitations as claim 2. (The 3D reconstruction instructions merely implement the depth 
recovery method of Claim 2). Therefore, with regard to claim 19, remarks analogous to 
those presented above relative to claim 2 are applicable. 

28. The following is in regard to Claim 3. As shown above the methods of [Chen, 
VSMM99] and [Zhang94] can be combined to satisfy the limitations of Claim 2. 
[Zhang94] further disclose that the "outlier-less" set of correspondences (i.e. the 
"second potential match set") is based at least in part on a least median of squares 
computation {LMedS - [Zhang94] Abstract and page 17, lines 4-7 and paragraph 2) of 
the reprojection errors (i.e. residuals n- see above) related to matched pixels in the first 
potential match set. LMedS, as put forth by [Zhang94], has an advantage over other 
methods in that it "is very robust to false.matches as well as outliers due to bad 
localization" ([Zhang94] page 1 7, paragraph 1 , last sentence). Therefore, it would have 
been obvious to one of ordinary skill in the art, at the time of the applicant's claimed 
invention, to use the LMedS of [Zhang94] to detect and remove outliers from the first 
potential match set. 

29. The following is in regard to Claim 20. Claim 20 recites essentially the same 
limitations as claim 3. (The 3D reconstruction instructions merely implement the depth 
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recovery method of Claim 3). Therefore, with regard to claim 20, remarks analogous to 
those presented above relative to claim 3 are applicable. 

30. Claims 10-1 1 and are rejected under 35 U.S.C. 103(a) as being unpatentable 
over [Chen, VSMM99], in view of [Okutomi93] (Okutomi and Kanade "A Multiple- 
Baseline Stereo", IEEE 1993), in further view of [Lewis95] (Lewis, "Fast Normalized 
Cross-Correlation", 1995). 

31 . The following is in regard to Claim 10. As shown above, [Chen, VSMM99] 
discloses a volumetric stereo matching method that sufficiently conforms to the depth 
recovery method proposed in Claim 9. As discussed above, the method of [Chen, 
VSMM99] involves the computation of the NCC between a first image window of a pre- 
determined size in the base view, and a second image window of the same pre- 
determined size in other views ([Chen, VSMM99] Section 3.1). It was also show above 
that the NCC is intimately related to the matching cost. Despite this, [Chen, VSMM99] 
does not expressly show or suggest that the matching cost is determined by computing 
the summation of all the normalized cross-correlations between a first image window of 
a pre-determined size in the base view, and a second image window of the same pre- 
determined size in a reference view. 

32. [Okutomi93] disclose a stereo matching method that uses multiple stereo pairs 
with various baselines for the purposes of depth recovery ([Okutomi93] Abstract 
sentence 1). According to [Okutomi93], the summation of the sum of squared 
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differences (SSD) from multiple stereo pairs can be used to indicate the "correctness" of 
a set of matching points (pixels). See [Okutomi93] page 353 (right column, paragraph 3, 
first and last sentence), page 354, (left column, lines 7-10), page 355 (Section B, 
paragraph 2, sentence 1), and equation (39) on page 362. This summation is analogous 
to a matching cost in that it approaches a minimum for correct matches ([Okutomi93] 
page 353, right column, paragraph 3, last sentence). According to equation (39) the 
computation is made within a predetermined window W belonging to each of the stereo 
images ([Okutomi93] page 354, right column, paragraph 1, line 4). In summary, 
[Okutomi93] shows a "matching cost" determined by computing the summation of all the 
SSD's between a first image window of a pre-determined size in the base view and a 
second image window of the same pre-determined size in other views (e.g. "reference 
views"). Given this, it would have been obvious to one of ordinary skill in the art, at the 
time of the applicant's claimed invention, to use a "matching cost" (i.e. a measure of the 
similarity or "correctness" of a match) determined by computing the summation of all 
the SSD's between a first image window of a pre-determined size in the base view and 
a second image window of the same pre-determined size in other views (e.g. "reference 
views"). As pointed out by [Okutomi93], this would have the effect of advantageously 
reducing global mismatches ([Okutomi93], page 353, right column, paragraph 3, first 
sentence). 

33. Note that the SSD and NCC are closely related mathematically. As measures of 
similarity among regions of images, they are functionally equivalent. See [Lewis95] 
Section 2, paragraph 1. Furthermore, it can also be shown that the process of 
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minimizing the SSD and maximizing the NCC are in fact equivalent operations. All of 
this suggests that SSD and NCC are essentially interchangeable. Moreover, the NCC 
has the advantage over the SSD in that it is invariant to "changes in image amplitude 
such as those caused by changing lighting conditions across the image sequence" 
([Lewis95] page 2, lines 8-17). Since lighting conditions are likely to vary across the 
various stereo images, it would have been obvious to one of ordinary skill in the art, at 
the time of the applicant's claimed invention, to substitute the NCC for the SSD in the 
stereo matching method of [Okutomi93]. This would yield a "matching cost" determined 
by computing the summation of all the NCC's between a first image window of a pre- 
determined size in the base view and a second image window of the same pre- 
determined size in other views (e.g. "reference views"). 

34. The teachings of [Chen, VSMM99] and [Okutomi93] are combinable because 
they are analogous art, namely, the art of stereo matching. The teachings of [Lewis95] 
are applicable to both the methods of [Chen, VSMM99] and [Okutomi93] because all 
relate generally to correlative approaches for evaluating the similarity of windowed- 
regions belonging to a collection of images of a scene. Combining these teachings as 
suggested above yields a method that conforms sufficiently to that of claim 10. 

35. The following is in regard to Claim 1 1. As shown above the teachings of [Chen, 
VSMM99], [Okutomi93], and [Lewis95] can be combined so a to yield a method that 
adequately satisfies the limitations of Claim 10. [Chen, VSMM99] suggests rectification 
of at least one pair of images corresponding to the base view of the scene and at least 
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one of the other views (e.g. "reference views"). See [Chen, VSMM99] Section 2. 
Therefore, it would have been obvious to one of ordinary skill in the art, at the time of 
the applicant's claimed invention, to further perform rectification of at least one pair of 
images corresponding to the base view of the scene and at least one of the other views 
(e.g. "reference views"). The result of rectification is that "scanlines [epipolar lines] are 
parallel in each image, and the corresponding ones are collinear across the images" 
([Chen, VSMM99] Section 2, paragraph 1 , sentence 2 and last paragraph). This is 
known to have the effect of greatly simplifying the problem of finding correspondences 
across stereo images. 

Allowable Subject Matter 

Objections, Allowable Subject Matter 

36. Claim 29 is found to be allowable in view of the presented amendment and 
discussion in view of the previously applied 1 12 rejection. 

37. The following is a statement of reasons for the indication of allowable subject 
matter. As shown above, [Chen, VSMM99] discloses: 

(29.a.) Tracing at least one parameter surface (e.g. disparity surface), 

each of the at least one parameter surface traced starting from at 
least one predetermined seed pixel point. 
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However, no prior art methods were encountered that involved or even suggested the 
step of: 

(29.b.) Calculating a derivative of function E(g) with respect to parameter g 
by using finite difference to minimize the equation of E(g) put forth 
in Claim 29. 



Conclusion 

Applicant's amendment necessitated the new grounds of rejction presented in 
the Office Action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Wes Tucker whose telephone number is 703-305-6700. 
The examiner can normally be reached on 9AM-5PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 571-272-7427. The fax phone number for the 
organization where this application or proceeding is assigned is 571-272-7414. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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